Background

Non-inferiority (NI) trials are increasingly used in clinical research^1,2. However, they are frequently poorly designed and interpreted. Common sources of confusion include the specification of NI margins, selection of appropriate active controls, and interpretation of statistical conclusions, which can lead to adverse consequences for manufacturers, clinicians, and the wider public.

What is a Non-Inferiority Trial?

• Non-Inferiority Trials: Clinical studies designed to demonstrate a new treatment is not clinically worse than an active control by more than a pre-specified margin³.

• Non-Inferiority Margin (Δ): Pre-specified & approved threshold the new treatment must meet to prove it preserves a clinically meaningful portion of the active control’s effect².

• NI Trials typically run like a randomised control trial but comparing the new treatment with an active control- an established standard of care used, instead of a placebo.

• NI must show the new treatment’s estimated effect, along with its confidence interval, lies within the pre-specified NI margin.

Case-study: Biomimics 3D stent

It is 2008 and we have just been hired as statisticians for BioMimics 3D Vascular Stent’s pivotal NI trial:

The BioMimics 3D stent is a peripheral vascular stent implanted in the leg or arm to improve blood flow in patients with peripheral vascular disease (narrowing of the peripheral blood vessels). Unlike conventional straight stents, it features a three-dimensional helical design, intended to improve vascular performance and blood flow in affected vessels.

From a statistical perspective, this study is not a traditional two-arm NI trial, but a single-arm trial evaluated against a fixed Performance Goal (PG). In medical device development, particularly For implantable medical devices, two-arm non-inferiority trials are often impractical due to cost, extended timelines, and challenges with recruitment and blinding. In such settings, regulators may accept a single-arm design, provided that the device’s observed performance, together with its confidence interval, exceeds a pre-specified PG within an agreed non-inferiority margin.

The PG and corresponding NI margins are derived from a random-effects meta-analysis of safety and effectiveness outcomes from existing, approved comparator devices. For safety, the margin is set at the upper bound of the 95% confidence interval, representing the maximum acceptable level of harm. For efficacy, the margin is set at the lower bound of the 95% confidence interval, representing the minimum clinical benefit that must be preserved. Crossing either bound results in failure to demonstrate NI.

Our project involves working with BioMimics’ lead scientist, to design, assess, and interpret this trial from a statistician’s perspective. This includes:

• Defining and justifying appropriate safety and efficacy endpoints.

• Ensuring the trial is statistically powered and ethically justified.

• Selecting valid analysis methods.

• Correctly interpreting and communicating non-inferiority conclusions.

Meta-Analysis & Evidence base

Prior to being available on the market all medical products must obtain approval from the relevant regulatory authority. This is done through a series of clinical studies where the medical product is monitored for safety and effectiveness.

Initially, a feasibility study is conducted. This is a small, single-arm prospective study for a narrowly defined patient population, designed to ensure preliminary safety and device performance. If successful, a pivotal clinical study is proposed. Our pivotal study is a single-arm objective performance goal study which requires pre-specified and approved primary efficacy and safety endpoints. These endpoints must be statistically justified, adequately powered, and grounded in clinical evidence of safety and benefit in the target population.

Evidence-Based Safety Performance Goals (30 Days)

Safety Endpoint: Must be able to identify potential harm, including any adverse events (AEs), serious adverse events (SAEs), lab abnormalities, or physical changes³.

Evidence-Based Efficacy Performance Goals (12 Months)

Efficacy: The treatment’s capacity to produce a desired, measurable, and beneficial therapeutic effect under ideal and strictly controlled conditions³.

To determine appropriate safety and efficiency endpoints for the Biomimics pivotal study we conducted a targeted literature review of clinical trials previously run involving peripheral vascular devices. This included a meta-analysis which we analysed & utilized at an individual participant data (IPD) level to justify our chosen endpoints^4–6.

Margin-Jinn

Margin-Jinn is an interactive Shiny application developed as part of this project to support statistical planning for NI trials. The app allows users to explore how sample size requirements vary under different design assumptions, including power, significance level, effect size, and choice of confidence-interval method.

Multiple approaches to interval estimation for proportions (e.g. Clopper–Pearson, Wilson, Agresti–Coull) are supported, enabling evaluation of how methodological choices influence NI conclusions, particularly in single-arm trials with small sample sizes and low event rates.

In this study, Margin-Jinn is applied to the BioMimics 3D case study to justify and critically assess sample size decisions under realistic design scenarios.

GitHub: https://github.com/FilipMKgit/Margin-Jinn

Next Steps

• Extend the BioMimics 3D case study through simulation of NI trial data under the defined Performance Goals.

• Finalise a Statistical Analysis Plan (SAP) specifying endpoints, estimators, confidence-interval methods, and NI decision rules prior to data unblinding.

• Analyse simulated trial outcomes to assess NI conclusions under alternative assumptions.

• Conduct sensitivity and tipping-point analyses to evaluate the robustness of conclusions to small changes in assumptions or observed event counts.